{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Serving LightGBM models\n",
    "\n",
    "Out of the box, `mlserver` supports the deployment and serving of `lightgbm` models.\n",
    "By default, it will assume that these models have been [serialised using the `bst.save_model()` method](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.Booster.html).\n",
    "\n",
    "In this example, we will cover how we can train and serialise a simple model, to then serve it using `mlserver`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training\n",
    "\n",
    "To test the LightGBM Server, first we need to generate a simple LightGBM model using Python. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import lightgbm as lgb\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn.model_selection import train_test_split\n",
    "import os\n",
    "\n",
    "model_dir = \".\"\n",
    "BST_FILE = \"iris-lightgbm.bst\"\n",
    "\n",
    "iris = load_iris()\n",
    "y = iris['target']\n",
    "X = iris['data']\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1)\n",
    "dtrain = lgb.Dataset(X_train, label=y_train)\n",
    "\n",
    "params = {\n",
    "    'objective':'multiclass', \n",
    "    'metric':'softmax',\n",
    "    'num_class': 3\n",
    "}\n",
    "lgb_model = lgb.train(params=params, train_set=dtrain)\n",
    "model_file = os.path.join(model_dir, BST_FILE)\n",
    "lgb_model.save_model(model_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our model will be persisted as a file named `iris-lightgbm.bst`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serving\n",
    "\n",
    "Now that we have trained and saved our model, the next step will be to serve it using `mlserver`. \n",
    "For that, we will need to create 2 configuration files: \n",
    "\n",
    "- `settings.json`: holds the configuration of our server (e.g. ports, log level, etc.).\n",
    "- `model-settings.json`: holds the configuration of our model (e.g. input type, runtime to use, etc.)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `settings.json`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile settings.json\n",
    "{\n",
    "    \"debug\": \"true\"\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `model-settings.json`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile model-settings.json\n",
    "{\n",
    "    \"name\": \"iris-lgb\",\n",
    "    \"implementation\": \"mlserver_lightgbm.LightGBMModel\",\n",
    "    \"parameters\": {\n",
    "        \"uri\": \"./iris-lightgbm.bst\",\n",
    "        \"version\": \"v0.1.0\"\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Start serving our model\n",
    "\n",
    "Now that we have our config in-place, we can start the server by running `mlserver start .`. This needs to either be ran from the same directory where our config files are or pointing to the folder where they are.\n",
    "\n",
    "```shell\n",
    "mlserver start .\n",
    "```\n",
    "\n",
    "Since this command will start the server and block the terminal, waiting for requests, this will need to be ran in the background on a separate terminal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Send test inference request\n",
    "\n",
    "We now have our model being served by `mlserver`.\n",
    "To make sure that everything is working as expected, let's send a request from our test set.\n",
    "\n",
    "For that, we can use the Python types that `mlserver` provides out of box, or we can build our request manually."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "x_0 = X_test[0:1]\n",
    "inference_request = {\n",
    "    \"inputs\": [\n",
    "        {\n",
    "          \"name\": \"predict-prob\",\n",
    "          \"shape\": x_0.shape,\n",
    "          \"datatype\": \"FP32\",\n",
    "          \"data\": x_0.tolist()\n",
    "        }\n",
    "    ]\n",
    "}\n",
    "\n",
    "endpoint = \"http://localhost:8788/v2/models/iris-lgb/versions/v0.1.0/infer\"\n",
    "response = requests.post(endpoint, json=inference_request)\n",
    "\n",
    "response.json()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see above, the model predicted the probability for each class, and the probability of class `1` is the biggest, close to `0.99`, which matches what's on the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_test[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
